Receiving device and receiving method

ABSTRACT

A decoding device that includes a decoding determination unit to determine a procedure of recovering and decoding missing packets in consideration of a packet missing pattern in data including a set of media packets and redundant packets generated by a two-dimensional XOR-based FEC encoding method. Further, a decoding unit executes the recovery of the missing packets according to the procedure determined by the decoding determination unit.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 15/323,841, filed on Jan. 4, 2017, which is a U.S.National Phase of International Patent Application No. PCT/JP2015/069556filed on Jul. 7, 2015, which claims priority benefit of Japanese PatentApplication No. JP 2014-147530 filed in the Japan Patent Office on Jul.18, 2014. Each of the above-referenced applications is herebyincorporated herein by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to a receiving device and a receivingmethod, and a computer program.

BACKGROUND ART

In recent years, there has been increased a demand of transmittingmultimedia data including video data, audio data, and the like in ashort delay via the Internet or another transmission line. For example,there exists a so-called remote operation application in which tworemote medical sites are connected via the Internet or the like, anoperation state is transmitted in a motion picture from one remoteoperation room, and a surgical instrument in the remote operation roomis operated while the video image is being watched in the other medicalsite. In this remote operation application, preferably the motionpicture is transmitted in a state in which the delay is not longer thanseveral frame intervals.

From the demand like this, for example, Patent Literature 1 proposes amethod of performing compression by the wavelet transform using everyseveral lines of each picture in a motion image as one compressionencoding block. In this method, without waiting for the input of thewhole data of the picture, compression and forward error correction(FEC) processing can be started, and, also when the compression data istransmitted through a network and display is performed on a receivingside, decoding processing can be started before the whole data of thepicture is received. Accordingly, in the technique disclosed in PatentLiterature 1, real-time motion picture transmission becomes available inthe delay not longer than the frame interval if network propagationdelay is sufficiently short.

CITATION LIST Patent Literature

Patent Literature 1: JP 2007-311924A

SUMMARY OF INVENTION Technical Problem

In the FEC encoding method, the receiving side can perform decoding intooriginal data by receiving a predetermined number of packets in aplurality of packets. In particular, a two-dimensional XOR-based FECencoding method is used for video transmission. In the two-dimensionalXOR-based FEC encoding method, however, recovery processing of a lostpacket is heavy, and system cost is increased, and the recovery of thelost packet takes time.

Accordingly, the present disclosure proposes a novel and improvedreceiving device, receiving method, and computer program capable ofexhibiting the maximum recovery performance by the minimum processing inthe two-dimensional FEC encoding method.

Solution to Problem

According to the present disclosure, there is provided a decodingdevice, including: a decoding determination unit configured to determinea procedure of recovering and decoding missing packets in considerationof a packet missing pattern in data including a set of media packets andredundant packets generated by a two-dimensional XOR-based FEC encodingmethod; and a decoding unit configured to execute the recovery of themissing packets according to the procedure determined by the decodingdetermination unit.

According to the present disclosure, there is provided a decodingmethod, including: determining a procedure of recovering and decodingmissing packets in consideration of a packet missing pattern in dataincluding a set of media packets and redundant packets generated by atwo-dimensional XOR-based FEC encoding method; and executing therecovery of the missing packets according to the determined procedure.

According to the present disclosure, there is provided a computerprogram causing a computer to execute: determining a procedure ofrecovering and decoding missing packets in consideration of a packetmissing pattern in data including a set of media packets and redundantpackets generated by a two-dimensional XOR-based FEC encoding method;and executing the recovery of the missing packets according to thedetermined procedure.

Advantageous Effects of Invention

According to the present disclosure as explained above, it is possibleto provide a novel and improved receiving device, receiving method,computer program capable of exhibiting the maximum recovery performanceby the minimum processing in the two-dimensional FEC encoding method.

Note that the effects described above are not necessarily limitative.With or in the place of the above effects, there may be achieved any oneof the effects described in this specification or other effects that maybe grasped from this specification.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an explanatory diagram showing an outline of an FEC method.

FIG. 2 is an explanatory diagram showing a redundant packet generationmethod in an XOR-based FEC encoding method.

FIG. 3 is an explanatory diagram showing a transmission state of an FECblock including media RTP packets and a redundant packet.

FIG. 4 is an explanatory diagram showing redundant packet generationmethods by one-dimensional and two-dimensional XOR-based FEC encodingmethods.

FIG. 5 is an explanatory diagram showing packet recovery by thetwo-dimensional XOR-based FEC encoding method.

FIG. 6 is an explanatory diagram showing a transmission systemconfiguration example according to an embodiment of the presentdisclosure.

FIG. 7 is an explanatory diagram showing a transmission systemconfiguration example according to an embodiment of the presentdisclosure.

FIG. 8 is a flow chart showing an operation example of a receivingdevice 20 according to an embodiment of the present disclosure.

FIG. 9 is an explanatory diagram showing recovery processing of missingpackets in the receiving device 20 according to the present embodiment.

FIG. 10 is an explanatory diagram showing recovery processing of missingpackets in the receiving device 20 according to the present embodiment.

FIG. 11 is an explanatory diagram showing recovery processing of missingpackets in the receiving device 20 according to the present embodiment.

FIG. 12 is an explanatory diagram showing recovery processing of missingpackets in the receiving device 20 according to the present embodiment.

FIG. 13 is an explanatory diagram showing recovery processing of missingpackets in the receiving device 20 according to the present embodiment.

DESCRIPTION OF EMBODIMENT(S)

Hereinafter, (a) preferred embodiment(s) of the present disclosure willbe described in detail with reference to the appended drawings. In thisspecification and the appended drawings, structural elements that havesubstantially the same function and structure are denoted with the samereference numerals, and repeated explanation of these structuralelements is omitted.

Here, explanation will be performed in the following order.1. Embodiment of the present disclosure1.1. Outline of two-dimensional FEC encoding method1.2. System configuration example1.3. Functional configuration example1.4. Operation example

2. Conclusion 1. Embodiment of the Present Disclosure 1.1. Outline ofTwo-Dimensional FEC Encoding Method

First, there will be explained an outline of a technique for an FECencoding method to be used in an embodiment of the present disclosure,in particular, a two-dimensional FEC encoding method.

As described above, there has been increased the demand of transmittingmultimedia data highly efficiently in a short delay via the Internet oranother transmission line. The multimedia data transmitted via a networkincludes data for a remotely controlled camera, a game machine, andremote medical care, for example.

Stream-type transmission which is one of multimedia data transmissionmethods needs various techniques. RTP (Realtime Transport Protocol)which is defined in IETF RFC 3550, for example, is mainly used as anInternet technique suitable for the stream-type transmission method.

The RTP does not guarantee data transmission in real time. Since thepriority, setting, management, and the like of packet delivery are notincluded in the transport service provided by the RTP, delivery delayand packet loss could occur in a media RTP packet as in the otherpackets. Here, the media RTP packet means an RTP packet configuring acontent data stream. Even when such delivery delay or packet lossoccurs, a receiving side can reproduce the data by using only packetswhich have arrived within an expected time. This is because video dataor audio data can be reproduced to some extent even when some data lossoccurs.

Here, a packet with delay delivery or a packet with error is discardedwithout any processing on the receiving side. That is, there is also aproblem that, even when high quality data is delivered, the data is notreproduced sufficiently well on the receiving side because of the packetloss or the error. In particular, under the condition in which it issaid that a wired section has error of 10⁻⁵ or more and a wirelesssection has error of 10⁻³ or more, reliability is low if the RTP isutilized as it is, from a point of keeping the quality of media to bedelivered.

From such a viewpoint, in the stream-type transmission method, there hasbeen developed a method of performing re-transmission request andre-transmission packet transmission using the TCP which has a highreliability in data transfer. However, since the TCP is robust to errorbut has a low throughput and a long delay, there is a possibility thatthe re-transmission packet does not arrive before reproduction time.

Since the RTP/UDP does not guarantee the video quality in the networktransmission, the QoS (Quality of Service) control becomes necessary.The QoS technique realizes stable transmission quality and high usersensory quality according to a network situation (packet loss rate ortransmission delay). Examples of the QoS technique to improve the datatransmission reliability by the use of the RTP include a method such asan automatic re-transmission method, a so-called ARQ (Auto RepeatreQuest) method, and a forward error correction encoding method(so-called FEC method).

The ARQ method detects a lost packet by utilizing the sequence number ofa media RTP packet, and requests the re-transmission of the lost packetfrom a receiving terminal to a transmission terminal. Further, the FECmethod uses a plurality of neighboring packets (having continuoussequence numbers) as one FEC block and performs redundant encoding usingXOR (eXclusive OR) calculation or using an error correcting code such asthe Reed-Solomon (RS) code.

FIG. 1 is an explanatory diagram showing an outline of the FEC method.As shown in FIG. 1, the FEC method generates and transmits one or moreredundant packets (packets shown by F1 and F2 in FIG. 1) from aplurality of data packets (packets indicated by 1 to 4 in FIG. 1) on thetransmitting side, and, if data packet missing is found on the receivingside (missing of the second packet is found, for example), recovers thedata packet using the redundant packets generated on the transmittingside.

The FEC method using the XOR calculation generates a redundant packet byXOR calculation of each bit in the packet. By the generation of theredundant packet by the XOR calculation of each bit in the packet, itbecomes possible to recover the loss of up to one packet in an FECblock. Here, the FEC block means a block composed of media RTP packetsto be used for the redundant packet generation and a redundant packet.

FIG. 2 is an explanatory diagram showing a redundant packet generationmethod in the XOR-based FEC encoding method. FIG. 2 shows an example ofgenerating the redundant packet using media RTP packets 1 to 3 eachcomposed of six bits, for simple explanation. As shown in FIG. 2, theredundant packet in the XOR-based FEC encoding method is generated byXOR of each bit in each packet.

FIG. 3 is an explanatory diagram showing how the FEC block includingmedia RTP packets and a redundant packet is transmitted. FIG. 3 showshow the FEC blocks each including a set of three media RTP packets and aredundant packet generated by the XOR calculation of the three media RTPpackets are sequentially transmitted.

The XOR-based FEC encoding method is utilized one-dimensionally andtwo-dimensionally. The FEC method using the XOR-based FEC encodingmethod is standardized in SMPTE (Society of Motion Picture andTelevision Engineers) 2022.

FIG. 4 is an explanatory diagram showing redundant packet generationmethods by the one-dimensional and two-dimensional XOR-based FECencoding methods. The redundant packets by the one-dimensional XOR-basedFEC encoding method are generated by the XOR calculation only for onedirection when a plurality of media RTP packets is arranged in n-rowsand m-columns as shown in FIG. 4. In FIG. 4, reference numbers 1 to 16indicate the RTC packets and reference symbols C1 to C4 indicate theredundant packets.

On the other hand, the redundant packets by the two-dimensionalXOR-based FEC encoding method are generated by the XOR calculation foreach of the two directions when a plurality of media RTP packets isarranged in n-rows and m-columns as shown in FIG. 4. That is, (n+m)redundant packets are generated by the two-dimensional XOR-based FECencoding method. In FIG. 4, reference numbers 1 to 16 indicate the mediaRTC packets, and reference symbols C1 to C4 and R1 to R4 indicate theredundant packets.

The two-dimensional XOR-based FEC encoding method is mainly used forvideo transmission. Accordingly, in an embodiment of the presentdisclosure to be explained in the following, the two-dimensionalXOR-based FEC encoding method is assumed to be used.

In the XOR-based FEC encoding method, when the packet missing occurs,only one packet is recovered by single recovery processing. In otherwords, it becomes not possible to perform recovery when error occurs intwo or more media RTC packets among a plurality of media RTC packetssubjected to the XOR operation.

The receiving side assumes that the media RTP packets are arrangedtwo-dimensionally, and executes the recovery processing using theredundant packets in either one of the vertical direction or thehorizontal direction. Then, when it is not possible to recover all themissing packets, next the receiving side executes the recoveryprocessing in the other direction. That is, the receiving side executesthe recovery processing first in the vertical direction, and, when it isnot possible to recover all the missing packets, next executes therecovery processing in the horizontal direction. Then, the receivingside executes the recovery processing while changing the direction untila recoverable packet does not exist.

FIG. 5 is an explanatory diagram showing the packet recovery by thetwo-dimensional XOR-based FEC encoding method. FIG. 5 shows the case ofgenerating the redundant packets in the vertical direction and thehorizontal direction from the media RTP packets of three rows and threecolumns and the media RTP packets of three rows and three columns, forsimple explanation.

As shown in the leftmost part of FIG. 5, it is assumed that media RTPpackets have arrived at the receiving side while media RTP packets ofNo. 3, No. 4, No. 5, No. 6, and No. 9 are missing. The receiving sideexecutes the recovery processing using the redundant packets in eitherone of the vertical direction or the horizontal direction, and, if it isnot possible to recover all the missing packets, next executes therecovery processing in the other direction.

In the example shown in FIG. 5, the receiving side first executes therecovery processing using the redundant packets in the horizontaldirection to recover the No. 3 and No. 9 packets, and subsequentlyexecutes the recovery processing using the redundant packets in thevertical direction to recover the No. 4, No. 5, and No. 6 packets.

In the example shown in FIG. 5, the recovery processing is executed twotimes in the horizontal direction and the vertical direction and therebyall the missing packets can be recovered. However, it is not possible toknow in advance how many times actually the recovery processing is to beexecuted for perfect recovery. Further, depending on a missing pattern,the missing packets sometimes cannot be recovered perfectly.Specifically, as explained by the use of FIG. 5, when the No. 5 mediaRTP packet is missing, the No. 5 media RTP packet cannot be recovered ifboth of the redundant packet C2 in the vertical direction and theredundant packet R2 in the horizontal direction are missing.

In this two-dimensional XOR-based FEC encoding method, the recoveryprocessing of the missing packet is heavy, thereby the system costbecomes high, and the recovery of the missing packet takes time.

Accordingly, the present disclosing party has studied a technique ofexhibiting the maximum recovery performance on the receiving side whilereducing time taken for the recovery processing on the receiving side inthe XOR-based FEC encoding method. Then, as will be explained in thefollowing, the present disclosing party has reached the idea for atechnique of exhibiting the maximum recovery performance on thereceiving side while reducing time taken for the recovery processing onthe receiving side in the XOR-based FEC encoding method by determining apacket missing situation on the receiving side.

The outline of the technique used in an embodiment of the presentdisclosure has been explained in the above. Subsequently, there will beexplained a transmission system configuration example according to anembodiment of the present disclosure.

1.2. System Configuration Example

FIG. 6 is an explanatory diagram showing a transmission systemconfiguration example according to an embodiment of the presentdisclosure. In the following, there will be explained a transmissionsystem configuration example according to an embodiment of the presentdisclosure by the use of FIG. 6.

The transmission system according to an embodiment of the presentdisclosure shown in FIG. 6 includes a transmitting device 10 and areceiving device 20. The transmitting device 10 and the receiving device20 are connected to a network 1, and can communicate with each otherthrough the network 1.

The transmitting device 10 is configured including, for example, acapturing unit 12 to capture an image acquired by an imaging device 11,an encoder 13 to encode image data captured by the capturing unit 12, apacketizing unit 14 to packetize the image data captured by thecapturing unit 12 or data encoded by the encoder 13 into a media RTPpacket, an FEC unit 15 to perform forward error correction processing(FEC), and a transmitting unit 16 to transmit the data to the receivingdevice 20.

The capturing unit 12 captures a video signal sent to the transmittingdevice 10 in units of line block. When having captured the video signalin units of line block, the capturing unit 12 sends the captured signalto the encoder 13, or sends the captured signal directly to thepacketizing unit 14 when the captured signal is not to be encoded.

The encoder 13 encodes the signal captured by the capturing unit 12 inunits of line block, by a predetermined method. For example, the encoder13 may encode the signal in units of line block by a method ofperforming wavelet transform compression using every several lines ofeach picture in a motion image as one compression encoding block asdisclosed in above Patent Literature 1, but the encoding method of thepresent disclosure is not limited to such an example. The encoder 13 canencode the signal in units of line block by any encoding method if theencoding method can be applied to the above supposed network system. Theencoder 13 sends the encoded data to the packetizing unit 14.

The packetizing unit 14 packetizes the image data captured by thecapturing unit 12 or the data encoded by the encoder 13 into a media RTPpacket. The media RTP packet is an example of a media packet of thepresent disclosure. Specifically, the packetizing unit 14 packetizes thedata encoded by the encoder 13 into the media RTP packet by adding anRTP header to the encoded data. The packetizing unit 14 sends thegenerated media RTP packet to the FEC unit 15.

The FEC unit 15 performs FEC-encoding on the media RTP packet generatedby the packetizing unit 14. When having performed FEC-encoding on themedia RTP packet generated by the packetizing unit 14, the FEC unit 15sends the encoded data to the transmitting unit 16. In the presentembodiment, the FEC unit 15 generates a redundant packet by the abovetwo-dimensional XOR-based FEC encoding method.

According to a preliminarily determined FEC bock size (for example, oneFEC block is assumed to be configured with four media RTP packetsarranged in each of the vertical direction and the horizontaldirection), the FEC unit 15 accumulates the media RTP packets generatedby the packetizing unit 14 in the amount corresponding to an FEC bocksize and generates the redundant packets by the XOR method in thevertical direction and the horizontal direction.

The transmitting unit 16 transmits the data encoded by the FEC unit 15toward the receiving device 20 through the network 1.

The receiving device 20 is configured including a receiving unit 21 toreceive the data transmitted from the transmitting device 10 through thenetwork 1, an FEC decoding unit 22 to decode the data encoded by theforward error correction processing, a de-packetizing unit 23 tode-packetize the data decoded by the FEC decoding unit 22, a decoder 24to decode the data after the de-packetizing in the de-packetizing unit23, and a display processing unit 25 to output the data after thede-packetizing in the de-packetizing unit 23 or the data after thedecoding in the decoder 24 to a display 26. In the present embodiment,if packet missing occurs, the FEC decoding unit 22 recovers the packetusing a redundant packet generated by the two-dimensional XOR-based FECencoding method.

The receiving unit 21 receives the data transmitted from thetransmitting device 10 through the network 1. When having received thedata transmitted from the transmitting device 10, the receiving unit 21sends the received data to the FEC decoding unit 22.

The FEC decoding unit 22 performs decoding processing by the FEC methodusing the data received by the receiving unit 21. The data subjected tothe decoding processing by the FEC method in the FEC decoding unit 22 isonce returned back to the receiving unit 21, and then sent to thede-packetizing unit 23. The FEC decoding unit 22 arranges the media RTPpackets and the redundant packets according to the FEC block size. If apacket is not received, the FEC decoding unit 22 assumes the packet tobe a lost packet. The FEC decoding unit 22 performs the lost packetdetermination on the basis of the sequence number of the packet, forexample. The FEC decoding unit 22 may be provided with a buffer having apredetermined capacity and may arrange non-arranged packets according tothe sequence number.

Then, if the media RTP packets are missing, the FEC decoding unit 22executes recovery processing of missing packets using the media RTPpackets and the redundant packets. In the present embodiment, since theredundant packet is generated by the two-dimensional XOR-based FECencoding method, if media RTP packets are missing, the FEC decoding unit22 performs the XOR operation sequentially on normally received mediaRTP packets and redundant packets to recover the missing media RTPpackets.

The de-packetizing unit 23 executes re-configuration processing of thedecoded media RTP packet data. When having re-configured the decodedmedia RTP packet data, the de-packetizing unit 23 sends the data afterthe reconfiguration to the decoder 24 (to the display processing unit 25when the data is not decoded in the transmitting device 10).

The decoder 24 executes decoding processing of the data after thereconfiguration sent from the de-packetizing unit 23. When havingdecoded the data after the reconfiguration sent from the de-packetizingunit 23, the decoder 24 sends the decoded data to the display processingunit 25.

The display processing unit 25 executes processing of outputting thedata after the de-packetizing in the de-packetizing unit 23 or the dataafter the decoding in the decoder 24 to the display 26. For example, thedisplay processing unit 25 outputs the data after the de-packetizing inthe de-packetizing unit 23 or the data after the decoding in the decoder24 to the display 26 via a video output IF (Video OUT).

The transmission system shown in FIG. 6 performs a series of processing(image capturing, encoding and decoding, FEC processing, and displayprocessing) in the transmitting device 10 and the receiving device 20 inunits of picture (field frame). Then, the transmission system shown inFIG. 6 divides one picture into line blocks each grouping every severallines and this series of processing are performed in parallel, andthereby short-delay transmission is realized. Furthermore, thetransmitting device 10 may execute pipe-lined processing when performingthe processing in units of line block.

Further, in the processing by the FEC decoding unit 22 of the receivingdevice 20, the present embodiment determines a packet missing situationand executes appropriate recovery processing of missing packets. Bydetermining the packet missing situation and executing the appropriaterecovery processing of missing packets, the receiving device 20 canexhibit the maximum recovery performance while reducing the time takenfor the recovery processing of missing packets.

FIG. 7 is an explanatory diagram showing a detailed configurationexample of a transmission system according to an embodiment of thepresent disclosure. In the following, there will be explained a detailedconfiguration example of a transmission system according to anembodiment of the present disclosure by the use of FIG. 7.

In a transmission system shown in FIG. 7, an FEC decoding determinationunit 27 is added to the receiving device 20 shown in FIG. 6.Accordingly, processing of the FEC decoding determination unit 27 willbe explained here.

The FEC decoding determination unit 27 determines the missing situationof packets received by the receiving unit 21 and determines appropriaterecovery processing of missing packet. While specific processing in theFEC decoding determination unit 27 will be described below, the FECdecoding determination unit 27 confirms the missing situation byarranging the media RTP packets and the redundant packetstwo-dimensionally and determines a direction in which the recoveryprocessing is performed first to be a first direction.

Specifically, the FEC decoding determination unit 27 arranges the mediaRTP packets and the redundant packets two-dimensionally, and comparesthe number of rows and the number of columns each having one missingpacket and determines which is larger. Then, the FEC decodingdetermination unit 27 determines a procedure of the recovery processingof media RTP packet so as to determine a direction in which the numberof missing packets is larger to be the first direction.

The FEC decoding unit 22 executes the recovery processing according tothe procedure determined by the FEC decoding determination unit 27. Whena missing pattern is taken into consideration in the recovery of missingmedia RTP packets, the FEC decoding unit 22 first executes recoveryprocessing of the media RTP packet in the first direction, and, if allthe missing packets are not recovered, subsequently executes recoveryprocessing of the media RTP packet in a second direction perpendicularto the first direction. That is, the FEC decoding unit 22 executes therecovery processing of media RTP packet two times in the first directionand the second direction. The FEC decoding unit 22 performs the XORcalculation on the normally received media RTP packets and redundantpackets as described above as the recovery processing of media RTPpacket.

In the receiving device 20 shown in FIG. 7, the FEC decodingdetermination unit 27 arranges the media RTP packets and the redundantpackets two-dimensionally and confirms the missing situation todetermine the direction in which the recovery processing is performedfirst, and thereby the receiving device 20 according to the presentembodiment can exhibit the maximum recovery performance while reducingthe time taken for the recovery processing of missing packets.

In the above, the detailed configuration example of the transmissionsystem according to an embodiment of the present disclosure has beenexplained by the use of FIG. 7. Subsequently, an operation example ofthe receiving device 20 according to an embodiment of the presentdisclosure will be explained.

1.3. Operation Example

FIG. 8 is a flow chart showing an operation example of the receivingdevice 20 according to an embodiment of the present disclosure. In thefollowing, an operation example of the receiving device 20 according toan embodiment of the present disclosure will be explained by the use ofFIG. 8.

When receiving the data from the transmitting device 10, the receivingdevice 20 executes preliminarily determined initialization processingsuch as initialization of variables to be used for the processing (stepS101). When the initialization processing in step S101 has beencompleted, subsequently the receiving device 20 receives the media RTPpackets and the redundant packets from the transmitting device 10 in thereceiving unit 21 (step S102), and configures one FEC block with thereceived media RTP packets and redundant packets in the FEC decodingunit 22 (step S103).

Subsequently, when missing occurs in the media RTP packets configuringthe FEC block, the receiving device 20 determines whether or not tolimit the number of times for executing the recovery processing ofmissing packet (number of recovery times) (step S104). The determinationin step S104 is executed by the FEC decoding determination unit 27, forexample. In the determination whether or not to limit the number ofrecovery times, the receiving device 20 refers to preliminarilyregistered setting whether or not to limit the number of recovery times,for example.

When it is determined that the number of recovery times is not to belimited as the above determination result in step S104 (No in stepS104), the receiving device 20 executes the recovery processing whileswitching the direction each time in the horizontal, vertical,horizontal, . . . directions, or the vertical, horizontal, vertical, . .. directions until the case appears where it is not possible to recovera missing media RTP packet (step S105). The recovery processing in stepS105 is executed by the FEC decoding unit 22.

On the other hand, when it is determined that the number of recoverytimes is to be limited as the above determination result in step S104(Yes in step S104), the receiving device 20 subsequently determineswhether or not to consider the missing pattern in the recovery of themissing media RTP packet (step S106). The determination in step S106 isexecuted by the FEC decoding determination unit 27, for example. In thedetermination whether or not to consider the missing pattern in therecovery of the missing packet, the receiving device 20 refers topreliminarily registered setting (setting whether or not to consider themissing pattern), for example.

When it is determined that the missing pattern is not considered in therecovery of the missing media RTP packets as the above determinationresult in step S106 (No in step S106), the receiving device 20 executesthe recovery processing up to total three times in the order of thehorizontal, vertical, and horizontal directions, or the vertical,horizontal, and vertical directions as the recovery processing of themissing media RTP packet (step S107). The recovery processing in stepS107 is executed by the FEC decoding unit 22.

On the other hand, when it is determined that the missing pattern isconsidered as the result of the above determination result in step S106(Yes in step S106), subsequently the receiving device 20 calculates thenumber of rows or columns each having one missing media RTP packet andredundant packet, for the vertical direction and the horizontaldirection in an FEC block where the media RTP packets and the redundantpackets are arranged two-dimensionally (step S108). The calculation instep S108 is executed by the FEC decoding determination unit 27, forexample.

When having calculated the number of rows or columns each having onemissing media RTP packet and redundant packet in above step S108,subsequently the receiving device 20 compares the number of rows and thenumber of columns each having one missing packet and determines which islarger to determine a direction of a larger number to be a firstdirection (step S109). The determination in step S109 is executed by theFEC decoding determination unit 27, for example.

When the number of rows each having one missing packet is larger as theabove determination result in step S109, as the recovery processing ofthe lost media RTP packet, the receiving device 20 determines a recoveryprocedure in which the recovery processing is executed up to total twotimes in the order of the horizontal direction of the first directionand the vertical direction of the second direction perpendicular to thefirst direction, and executes the recovery processing according to thedetermined recovery procedure (step S110).

On the other hand, when the number of columns each having one missingpacket is larger as the above determination result in step S109, as therecovery processing of the missing media RTP packet, the receivingdevice 20 determines a recovery procedure in which the recoveryprocessing is executed up to total two times in the order of thevertical direction of the first direction and the horizontal directionof the second direction perpendicular to the first direction, andexecutes the recovery processing according to the determined recoveryprocedure (step S111). The recovery processing in step S110 or step S111is executed by the FEC decoding unit 22.

Note that, if the number of rows and the number of columns each havingone missing packet are the same as the above determination result instep S109, the receiving device 20 may determine a recovery procedure inwhich any direction is the first direction. That is, while, when thenumber of rows and the number of columns each having one missing packetare the same as the above determination result in step S109, thereceiving device 20 may execute the recovery processing first either inthe horizontal direction or in the vertical direction, the recoveryprocessing is assumed to be executed up to total two times also in thiscase. Obviously, if all the missing packets can be recovered in thefirst direction, the receiving device 20 needs not perform the recoveryprocessing in the second direction.

FIG. 9 to FIG. 12 are explanatory diagrams to explain the recoveryprocessing of missing packets in the receiving device 20 according tothe present embodiment. FIG. 9 to FIG. 12 show an example in which thenumber of media RTP packets composing one FEC block is 16, the media RTPpackets are arranged in four rows by four columns, and the redundantpackets are generated by the XOR calculation in each of the rows andeach of the columns.

In FIG. 9 to FIG. 12, No. 0 to No. 3, No. 5 to No. 8, No. 10 to No. 13,and No. 15 to No. 18 packets are media RTP packets, No. 4, No. 9, No.14, and No. 19 packets are redundant packets generated by the XORoperation of the media RTP packets in the horizontal direction, and No.20 to No. 23 packets are redundant packets generated by the XORoperation of the media RTP packets in the vertical direction.

FIG. 9 shows an example of the state in which the media RTP packets andthe redundant packets arrive at the receiving device 20. FIG. 9 showsthe state in which the No. 6, No. 8, and No. 12 media RTP packets andthe No. 21 redundant packet are missing. When the media RTP packets andthe redundant packets have arrived in this state, in above step S108,the receiving device 20 calculates the number of rows or columns eachhaving one missing media RTP packet or redundant packet for the verticaldirection and the horizontal direction of the FEC block. FIG. 10 showsthe state in which the number of rows or columns each having one missingmedia RTP packet or redundant packet is calculated.

As shown in FIG. 10, in this example, the number of rows each having onemissing media RTP packet or redundant packet is one and the number ofcolumns each having one missing packet is two. Accordingly, thereceiving device 20 executes the recovery processing up to total twotimes in the order of the vertical direction and the horizontaldirection, as the recovery processing of the missing media RTP packets.

FIG. 11 shows the state of the packet missing after the recoveryprocessing (XOR operation) has been executed in the vertical direction.The receiving device 20 can recover missing of the No. 8 and No. 12media RTP packets by executing the recovery processing (XOR operation)in the vertical direction. The missing of the No. 8 media RTP packet canbe recovered by the XOR operation executed sequentially for the No. 3,No. 13, No. 18, and No. 23 packets. Further, the missing of the No. 12media RTP packet can be recovered by the XOR operation executedsequentially for the No. 2, No. 7, No. 17, and No. 22 packets.

Then, FIG. 12 shows the state of the packet missing after the recoveryprocessing (XOR operation) has been executed in the horizontal directionafter the recovery of the missing of the No. 8 and No. 12 media RTPpackets as shown in FIG. 11. By executing the recovery processing in thehorizontal direction after executing the recovery processing in thevertical direction, the receiving device 20 can recover missing of theNo. 6 media RTP packet. The missing of the No. 6 media RTP packet can berecovered by the XOR operation executed sequentially for the No. 5, No.7, No. 8, and No. 9 packets. Accordingly, by executing the recoveryprocessing up to total two times in the vertical direction and thehorizontal direction, the receiving device 20 can recover missing of allthe media RTP packets.

There will be shown a case in which the receiving device 20 is set so asto execute the recovery processing (XOR operation) first in thehorizontal direction, for example, without calculating the number ofrows or columns each having one missing media RTP packet or redundantpacket.

FIG. 13 is an explanatory diagram to explain recovery processing ofmissing packets in the receiving device 20 according to the presentembodiment. In FIG. 13, the No. 0 to No. 3, No. 5 to No. 8, No. 10 toNo. 13, and No. 15 to No. 18 packets are media RTP packets, the No. 4,No. 9, No. 14, and No. 19 packets are redundant packets generated by theXOR operation of the respective media RTP packets in the horizontaldirection, and the No. 20 to No. 23 packets are redundant packetsgenerated by the XOR operation of the respective media RTP packets inthe vertical direction.

When the packets arrive at the receiving device 20 in the state in whichthe No. 6, No. 8, and No. 12 media RTP packets and the No. 21 redundantpacket are missing as shown in FIG. 9, if the recovery processing (XORoperation) is executed first in the horizontal direction, only the No.12 media RTP packet can be recovered as shown in FIG. 13.

By executing the recovery processing in the vertical directionsubsequently after having executed the recovery processing first in thehorizontal direction, the receiving device 20 can recover the No. 8media RTP packet as shown in FIG. 11. Then, by subsequently executingthe recovery processing further in the horizontal direction, thereceiving device 20 can recover missing of the No. 6 media RTP packet asshown in FIG. 12. Accordingly, the receiving device 20, when set so asto execute the recovery processing first in the horizontal direction, isunable to recover missing of all the media RTP packets without executingthe recovery processing up to total three times in the order ofhorizontal direction, vertical direction and horizontal direction.

That is, by calculating the number of rows or columns each having onemissing media RTP packet or redundant packet and executing the recoveryprocessing from a direction having a larger number of missing packets,the receiving device 20 according to an embodiment of the presentdisclosure can reduce the number of times of the recovery processingcompared with the case of executing the recovery processing withoutcalculating the number of rows or columns each having one missingpacket.

After having performed the recovery processing in each of above stepsS105, S107, S110, and S111, the receiving device 20 transfers one FECblock for which the recovery processing has been executed, to thedecoder 24 or the display processing unit 25 (step S112). Then, thereceiving device 20 continues the above series of processing as long asdata is being transmitted from the transmitting device 10.

The operation example of the receiving device 20 according to anembodiment of the present disclosure has been explained above by the useof FIG. 8. By executing the above series of operations, the receivingdevice 20 according to an embodiment of the present disclosure executesthe recovery processing for the vertical direction and the horizontaldirection in the FEC block in which packets are arrangedtwo-dimensionally, from a direction which includes a larger number ofrows or columns each having one missing media RTP packet or redundantpacket, and thereby can exhibit the maximum recovery performance whilereducing the time taken for the recovery processing of the missingpacket.

2. Conclusion

As explained above, an embodiment of the present disclosure provides thereceiving device 20 which determines the packet missing situation andexecutes appropriate recovery processing of the missing packet. Thereceiving device 20 according to an embodiment of the presentdisclosure, in advance of the recovery processing of missing packets,calculates the number of rows or columns each having one missing mediaRTP packet or redundant packet for the vertical direction and thehorizontal direction of the FEC block in which the packets are arrangedtwo-dimensionally. Then, the receiving device 20 according to anembodiment of the present disclosure executes the recovery processingfrom a direction including a larger number of missing packets.

By determining the packet missing packets situation and executing theappropriate recovery processing of missing packets, the receiving device20 according to an embodiment of the present disclosure can exhibit themaximum recovery performance while reducing the time taken for therecovery processing of missing packets.

Steps in processes executed by devices in this specification are notnecessarily executed chronologically in the order described in asequence chart or a flow chart. For example, steps in processes executedby devices may be executed in a different order from the order describedin a flow chart or may be executed in parallel.

Further, a computer program can be created which causes hardware such asa CPU, ROM, or RAM, incorporated in each of the devices, to function ina manner similar to that of structures in the above-described devices.Furthermore, it is possible to provide a recording medium having thecomputer program recorded thereon. Moreover, by configuring respectivefunctional blocks shown in a functional block diagram as hardware or ahardware circuit, the hardware or hardware circuit can achieve a seriesof processes.

Further, a part of or the whole part of each functional block shown inthe functional block diagram used in the above explanation may berealized by a server device connected via a network such as theInternet, for example. In addition, the configuration of each functionalbock shown in the functional block diagram used in the above explanationmay be realized by a single device or may be realized by a system inwhich a plurality of devices cooperates with one another. The system inwhich a plurality of devices cooperates with one another can include acombination of a plurality of server devices, a combination of a serverdevice and a terminal device, and the like, for example.

The preferred embodiment(s) of the present disclosure has/have beendescribed above with reference to the accompanying drawings, whilst thepresent disclosure is not limited to the above examples. A personskilled in the art may find various alterations and modifications withinthe scope of the appended claims, and it should be understood that theywill naturally come under the technical scope of the present disclosure.

Further, the effects described in this specification are merelyillustrative or exemplified effects, and are not limitative. That is,with or in the place of the above effects, the technology according tothe present disclosure may achieve other effects that are clear to thoseskilled in the art based on the description of this specification.

Additionally, the present technology may also be configured as below.

-   -   (1)

A decoding device, including:

a decoding determination unit configured to determine a procedure ofrecovering and decoding missing packets in consideration of a packetmissing pattern in data including a set of media packets and redundantpackets generated by a two-dimensional XOR-based FEC encoding method;and

a decoding unit configured to execute the recovery of the missingpackets according to the procedure determined by the decodingdetermination unit.

-   -   (2)

The decoding device according to (1), wherein

the decoding determination unit calculates the number of rows and thenumber of columns each having only one missing packet in the mediapackets and the redundant packets, and determines a procedure with adirection of a larger number as a first direction.

-   -   (3)

The decoding device according to (2), wherein

the decoding unit executes recovery of missing packets in the firstdirection in the procedure determined by the decoding determination unitand recovery of missing packets in a second direction perpendicular tothe first direction.

-   -   (4)

The decoding device according to (2), wherein

the decoding determination unit determines a procedure with anydirection as a first direction when the number of rows and the number ofcolumns each having only one missing packet are the same.

-   -   (5)

The decoding device according to any of (1) to (4), wherein

the redundant packets are generated by performing XOR operationprocessing on each of the media packets configured in a plurality ofrows and columns, for each of the rows and columns.

-   -   (6)

A decoding method, including:

determining a procedure of recovering and decoding missing packets inconsideration of a packet missing pattern in data including a set ofmedia packets and redundant packets generated by a two-dimensionalXOR-based FEC encoding method; and

executing the recovery of the missing packets according to thedetermined procedure.

-   -   (7)

A computer program causing a computer to execute:

determining a procedure of recovering and decoding missing packets inconsideration of a packet missing pattern in data including a set ofmedia packets and redundant packets generated by a two-dimensionalXOR-based FEC encoding method; and

executing the recovery of the missing packets according to thedetermined procedure.

REFERENCE SIGNS LIST

-   1 network-   10 transmitting device-   11 imaging device-   12 capturing unit-   13 encoder-   14 packetizing unit-   15 FEC unit-   16 transmitting unit-   20 receiving device-   21 receiving unit-   22 FEC decoding unit-   23 de-packetizing unit-   24 decoder-   25 display processing unit-   26 display-   27 FEC decoding determination unit

What is claimed is:
 1. An information processing device, comprising:circuitry configured to: calculate each of a number of rows and a numberof columns with one missing packet of a plurality of media packets ofdata, wherein generation of the data is based on a two-dimensionalXOR-based forward error correction (FEC) encoding process; compare thecalculated number of rows with the calculated number of columns;determine a first recovery process of a plurality of recovery processesfor the one missing packet; and execute the determined first recoveryprocess in one of a row direction or a column direction based on thecomparison.
 2. The information processing device according to claim 1,wherein the first recovery process is executed in the row directionbased on the calculated number of rows that is larger than thecalculated number of columns, and the first recovery process is executedin the column direction based on the calculated number of columns thatis larger than the calculated number of rows.
 3. The informationprocessing device according to claim 1, wherein the circuitry is furtherconfigured to: determine a number of recovery times is one of limited ornot limited; determine the first recovery process based on thecomparison and the determination that the number of recovery times islimited; and determine, based on the determination that the number ofrecover times is not limited, the first recovery process independent ofthe comparison.
 4. The information processing device according to claim1, wherein the circuitry is further configured to determine, based onthe calculation, a second recovery process from the plurality ofrecovery processes for the one missing packet.
 5. The informationprocessing device according to claim 4, wherein the circuitry is furtherconfigured to: execute, based on the calculation, the first recoveryprocess in the row direction; and execute the second recovery process inthe column direction based on the execution of the first recoveryprocess in the row direction, wherein the column direction isperpendicular to the row direction.
 6. The information processing deviceaccording to claim 1, wherein the circuitry is further configured todetermine a direction of execution of the first recovery process as oneof the row direction or the column direction based on the calculatednumber of rows that is same as the calculated number of columns.
 7. Theinformation processing device according to claim 1, wherein the dataincludes a plurality of redundant packets, and generation of theplurality of redundant packets is based on execution of thetwo-dimensional XOR-based FEC encoding process on each media packet ofthe plurality of media packets in a plurality of rows and a plurality ofcolumns.
 8. A method, comprising: calculating each of a number of rowsand a number of columns with one missing packet of a plurality of mediapackets of data, wherein generation of the data is based on atwo-dimensional XOR-based forward error correction (FEC) encodingprocess; comparing the calculated number of rows with the calculatednumber of columns; determining a first recovery process of a pluralityof recovery processes for the one missing packet; and executing thedetermined first recovery process in one of a row direction or a columndirection based on the comparison.
 9. The method according to claim 8,wherein the first recovery process is executed in the row directionbased on the calculated number of rows that is larger than thecalculated number of columns, and the first recovery process is executedin the column direction based on the calculated number of columns thatis larger than the calculated number of rows.
 10. The method accordingto claim 8, further comprising: determining a number of recovery timesis one of limited or not limited; determining the first recovery processbased on the comparison and the determination that the number ofrecovery times is limited; and determining, based on the determinationthat the number of recover times is not limited, the first recoveryprocess independent of the comparison.
 11. The method according to claim8, further comprising determining, based on the calculation, a secondrecovery process from the plurality of recovery processes for the onemissing packet.
 12. The method according to claim 11, furthercomprising: executing, based on the calculation, the first recoveryprocess in the row direction; and executing the second recovery processin the column direction based on the execution of the first recoveryprocess in the row direction, wherein the column direction isperpendicular to the row direction.
 13. The method according to claim 8,further comprising determining a direction of execution of the firstrecovery process as one of the row direction or the column directionbased on the calculated number of rows that is same as the calculatednumber of columns.
 14. The method according to claim 8, wherein the dataincludes a plurality of redundant packets, and generation of theplurality of redundant packets is based on execution of thetwo-dimensional XOR-based FEC encoding process on each media packet ofthe plurality of media packets in a plurality of rows and a plurality ofcolumns.
 15. A non-transitory computer-readable medium having storedthereon computer-executable instructions that, when executed by aprocessor, cause the processor to execute operations, the operationscomprising: calculating each of a number of rows and a number of columnswith one missing packet of a plurality of media packets of data, whereingeneration of the data is based on a two-dimensional XOR-based forwarderror correction (FEC) encoding process; comparing the calculated numberof rows with the calculated number of columns; determining a firstrecovery process of a plurality of recovery processes for the onemissing packet; and executing the determined first recovery process inone of a row direction or a column direction based on the comparison.16. The non-transitory computer-readable medium according to claim 15,wherein the first recovery process is executed in the row directionbased on the calculated number of rows that is larger than thecalculated number of columns, and the first recovery process is executedin the column direction based on the calculated number of columns thatis larger than the calculated number of rows.
 17. The non-transitorycomputer-readable medium according to claim 15, further comprising:determining a number of recovery times is one of limited or not limited;determining the first recovery process based on the comparison and thedetermination that the number of recovery times is limited; anddetermining, based on the determination that the number of recover timesis not limited, the first recovery process independent of thecomparison.
 18. The non-transitory computer-readable medium according toclaim 15, further comprising determining, based on the calculation, asecond recovery process from the plurality of recovery processes for theone missing packet.
 19. The non-transitory computer-readable mediumaccording to claim 18, further comprising: executing, based on thecalculation, the first recovery process in the row direction; andexecuting the second recovery process in the column direction based onthe execution of the first recovery process in the row direction,wherein the column direction is perpendicular to the row direction. 20.The non-transitory computer-readable medium according to claim 15,further comprising determining a direction of execution of the firstrecovery process as one of the row direction or the column directionbased on the calculated number of rows that is same as the calculatednumber of columns.